CLAIMS 

\ 

1 1\^ A computer program product for compressing data 

2 files representative of an image document having color 

3 information and/or graphical information, said software 

4 product disposed on a computer readable medium comprising 

5 instructionsS^for causing a computer to: 

6 provide a first image file at a first resolution and a 

7 second image file at a second resolution of said document 

8 with said second\resolution being lower than said first 

9 resolution; \ 

f^O process the first image file to convert the first image 

^kl file into a text file^ representation of the document; 

yy \ 

*§2 compress said' text file representation of the document 

y3 \ 

r|3 to provide a first compressed file; 

^4 process said second file to extract information from 

el 5 the image representation ok the document corresponding to 

£6 color information and graphrcs information; 
IJ7 compress the second file\using a second compression 

**j|8 technique to provide a second compressed file containing 

19 information corresponding to the image; and 

20 store said first and second ^compressed files and color 

21 information to provide a composite\ compressed file 

22 corresponding to the document. \ A 

X 

1 2. The computer program product\as recited in claim 1 

2 wherein said instruction for causing the computer to provide 

3 the image file comprises instructions for\ causing the 

4 computer to scan the document at a first resolution to 

5 provide said first image file at said first ^resolution and 

6 scan the document at a second resolution to provide said 
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7 second image file at said second resolution. 

1 3. \ The method as recited in conjunction with claim 2 

2 wherein said instruction to process the second file to 

3 extract information corresponding to color information 

4 comprises instructions for causing the computer to: 

5 determine foreground colors corresponding to colors 

6 associated witV^text portions of the document. 

1 4 . The computer program product as recited in 

p 2 conjunction with claim 3 further comprising instructions for 

^ 3 causing the computer to: 

«j 4 determine the foreground colors by causing the computer 

□ 5 to retrieve a plurality of samples of groups of pixels from 

rj \ 

m 6 the low resolution image representation of the document and 

jL 7 from each one of said samples of pixels finding a pixel 

yi 8 corresponding to the mirk mum and maximum intensity of the 

SJ 9 pixels in the sample; \f\ 
.^LO and for each one of said samples 

11 calculate a threshold value representative of the 

12 document by averaging the mAiimum and maximum intensities 

13 for each of the blocks; \ 

14 determine a color associated with each one of the 

15 blocks and the width of intensity of each one of the blocks 

16 and \ 

17 provide a data structure hav\ng an entry for each one 

18 of said blocks corresponding to a foreground color and a 

19 background color. \ 
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1 \5 . The computer program product as recited in 

2 conjunction with claim 1 further comprising instructions for 

3 causing\the computer to detect and remove data in said image 

4 representation of the document which when viewed on a 

5 display dfepicts a crease over a portion of the document. 



1 6. The computer program product as recited in 

2 conjunction With claim 5 comprises instructions for causing 

3 the computer to: 

4 form a first mask to apply to the text representation 
OS of a document to\mask out portions of said representation 
Q6 corresponding to Vmage portions of the document; and 

/ij7 provide a second mask to apply to the low resolution 

Q8 image portion of the document to mask out portions of said 

: : \ 
~f \ 

g|9 document corresponding to text portions of the document. 



rfl 7 • A computer system including a computer software 

i2 product for compressing data files representative of an 

rj 3 image document, said document including color information 

4 and/or graphical information, said computer system 

5 including: \ 

6 a processor to execute said software instructions; 

7 a memory storing said software program; 

8 a display which displays^ representations of said 

9 document ; 

10 said software product disposed on a computer readable 

11 medium comprising instructions for causing a computer to: 

12 provide a first image file at a first resolution 

13 and a second image file at a seconcK resolution of said 



ti 
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14 dbcument with said second resolution being lower than said 

15 first resolution; 

16 \ process the first image file to convert the first 

17 image\file into a text file representation of the document; 

18 \ compress said text file representation of the 

19 document\ to provide a first compressed file; 

20 \process said second file to extract information 

21 from the image representation of the document corresponding 

22 to color information and graphics information; 

23 compress the second file using a second 

24 compression technique to provide a second compressed file 
^5 containing information corresponding to the image; and 

- \ 

^26 store said first and second compressed files and 

tf27 said color information to provide a composite compressed 

Cp8 file of the document. 

I \A 

;L 1 8 . The computer system as recited in claim 7 wherein 

kJ \ 

yg 2 said instruction for causing the computer to provide the 

%j 3 first file comprises instructions for causing the computer 

^ 4 to scan the document at a first resolution to provide said 

5 first image file at said Dirst resolution and scan the 

6 document at a second resolution to provide said second image 

7 file at said second resolution. 

1 9. The computer system aaa recited in conjunction with 

2 cla.im 8 wherein said instructions, for causing the computer 

3 to process the second file to extract information 

4 corresponding to color information comprises instructions 

5 for causing the computer to: \ 

6 determine foreground color corresponding to colors 
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associated with text portions of the document. 

i0. The computer system as recited in conjunction with 
claim 9y further comprising instructions for causing the 
computeA to 

determine the foreground colors by causing the computer 
to retrieve, a plurality of sample of pixels from the low 
resolution itnage representation of a document and from each 
one of said samples of pixels finding a pixel corresponding 
to the minimum \and maximum intensity of the pixels in the 
sample; 

and for each\one of said samples 

calculate a threshold value representative of the 
document by averaging the minimum and maximum intensities 
for each of the blocks,* 

determine a colorYassociated with each one of the 
blocks and the width of\Lntensity of each one of the blocks; 
and 

provide a color data structure having an entry for each 
one of said blocks corresponding to a foreground color and a 
background color. 



11. The computer system as recited in conjunction with 
claim 7 further comprising instructions for causing the 
computer to: 

detect and remove data in said\image representation of 
the document which when viewed on a display forms a crease 
over a portion of the document. 

12. The computer system as recited ifci conjunction with 
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claim 11 comprises instructions for causing the computer to: 

form a first mask to apply^tr&the text representation 
of a document to mask ouTVpprtions of said representation 
corresponding to imag£ potions of the document; and 

provide a secop^masj^l^^ply to the low resolution 
image portion o^t he ^doisrument to mask out portions of said 
document corresponding to text portions of the document. 

^ ^ A computer program product operable to determine a 
dominant background color associated with an image 
representation of a document comprises instructions for 
causing a computer to: 

retrieve background color information associating a 
background color with each one of a plurality of samples of 
pixels representing the document; 

filter said background colors to provide a target 
number of colors to represent the background colors; 

apply a median cut analysis to the background color 
samples to filter said background samples into one of a 
plurality of boxes corresponding to said target number of 
colors; 

sort said boxes by increasing volume; 

sort a first portion of said boxes having the smallest 
amount of volume by decreasing intensity; and 

determine the dominant background color as a color to 
represent the background of the document by the box having 
the lowest intensity. 

/</ 

/ 

1£ . A computer program product for decompressing a 
file containing image information and text information, said 



# 



program residing on a computer readable medium comprising 
instructions fox causing a computer to: 

decompress the file containing image information and 
text information into an image file and a text file; 

allocate a target bit map to represent the decompressed 

file; 

insert the decompressed image information into the 
target bit map at locations specified by information 
contained in said file containing image information and text 
information; and 

insert text information into said target bit map in 
accordance with positional information provided from the 
decompressed text file. 



T" 5 ■ 

yf. The computer program product of claim lS^ftirther 
comprising instructions for causing a computep^xo display 
the reconstructed representation of the decompressed file on 
a computer monitor. 

(c 

1.8. The computer program p/b&6ct of claim 16 further 



comprising instructions for cai/sing 



a computer to fill the 



target output bit map with a cpyorj porre3ponding to a 
dominant background color provkdeKi^f rom color information in 
the file. 

l.sf. The computer program product as recited in 
conjunction with claim 18 further comprising instructions 
for causing a computer /to apply a color to the text 
information in the taxget bit map in accordance with said 
color information^provided from said file. 



6 2ef. A computer system including a computer Software 

7 product, said computer system including; 

8 a processor to execute said software instructions; 

9 a memory storing said software progranf; 

10 a display which displays representations of said 

11 document, for compressing data files representative of an 

12 image document, said document including color information 

13 and/or graphical information; 

14 said system operable to determine a dominant background 

15 color associated with an image representation of a document 

16 containing color and text information comprises instructions 
^7 for causing the computer to: j 

^8 retrieve background color information associating a 

yB9 background color with each one /of a plurality of samples of 
y3 / 

g|0 pixels representing the document; 

W» filter said background colors to provide a target 

* : / 

J22 number of colors to represent the background colors; 
y|3 apply a median cut arfal?ysis on the background color 

i2]4 samples to assign said background samples into one of a 

%S plurality of three dimeLsi^x^A^ boxes^in R-B-G color space, 

26 said plurality correspon^iWto said target number of 

27 colors; and j 

28 determine the dominant background color as a color to 

29 represent the background of the document by the three 

30 dimensional box having/the lowest intensity. 

1 2s£. The computer system of claim 2 0 wherein the 

2 computer program product further includes instructions for 

3 causing the computer to 

4 sort said three dimensional boxes by increasing volume; 
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and 

sort a first portion of said three dimensionajKboxes 
having the smallest amount of volume by decreasing intensity 
to determine. the dominate background color. 

22. The computer of claim 21 wherein the computer 
program product further includes instructions for causing 
the computer to sort the three dimensional boxes in R-G-B 
space by increasing distance (D) from the dominant 
background color determined by: 

D = sqrt((Rl - R2) * (Rl - R2) + (Gl - G2) * (Gl - G2) 



+ (Bl - B2) A (Bl - B2) ) . 

/ 
/ 

23. The computer of claim 21 wherein the computer 

/ . 

program product further includes instructions for causing 

/ 

the computer to estimate a width of the dominate backgroud 

color by using a distance to one of the boxes in the first 

* 

portion of boxs as thew\dth estimate. 



The compujber/of claim 21 wherein the computer 
program product further /mcludes instructions for causing 
the computer to map \q0X9rs to the dominate background color. 



2S. A computer program product for a reading machine 
to read words from/ a document allowed to a user and residing 
on a computer readable medium comprising instructions for 
causing a computer to: 

decompress a file comprising image and text into an 
image file and text file; 

reconstruct the document from the decompressed image 
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file and text file; 

apply color to the text in accordance with ceflor 
information provided from the file comprising^lmage and text 
and positional information provided from s^id text file; 

display the reconstructed image representation of the 
document on a computer monitor; and 

manipulate the displayed reconstructed image 
representation of the document by /using positional 
information in the text file. 

+1 / 

26. The computer program' product of claim 25 further 



comprising instructions for causing the computer to scale 
the positional information in the text file for manipulation 
of the image in accordance with a resolution of the image 
file and a resolution of the text file. 



ct a document item from a 
^presentation of the 



2-7. The computer program product as recited in 
conjunction with claim /#6J\wherein said program further 
comprises instructions for\causing the computer to: 

permit the user 
displayed reconstruct 

document ; and 

/ 

apply the converted text representation corresponding 
to the document item to a speech synthesizer to cause the 
computer to output a/synthesized speech representation of 
the document item. 




1 
2 
3 



2-8 . The computer program product as recited in 
conjunction wit'h claim 27 further comprising computer 
instructions/for causing the coumputer to display image 
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representation of the document to be highlighted by>£pplying 
a highlighting indicia to the displayed reconstructed image 
representation in accordance with scaled positional 
information provided from the text file, 

29. The computer program product /of claim 28 further 
comprising instructions for causing a/computer to dissplay 
an image representation of a document item to be highlighted 
with a color by applying a color to the displayed image 
representation in accordance with' positional information 
provided from the converted text/ file 

a* 

30 y . A method of compressing an image representation of 
a document having color portions and text portions comprises 
the steps of: 



scanning a document, 
resolution and a second file 
converting the fijrst 
applying an auto- 



provide a first file at a first 
at a second, lower resolution; 
le into/ text file; 



:e fiTter to the first file to 
correct said file for e^brs; 

converting sai$Miifcfh resolution image file into an 
optical character recognition file having text information 
and positional information corresponding to the text 
information on the image document; 

masking portions of said optical character recognition 
file corresponding to portions of said document representing 
graphical information associated with the document; and 

compressing the unmasked portions of said optical 
character Recognition file to provide a compressed text 
file; 
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19 applying a rotate filter to the second file to cprf^ct 

20 errors in said second file; y< 

21 determining from said second file foregrotond colors 

22 associated with each of the sections of said document and 

23 background colors associated with each portion of said 

24 document; / 

25 determining from said background^ colors a dominant 

26 background color that best represents the background color 

27 of the document; j 

2 8 masking portions of said document not corresponding to 

29 the graphical portions of the document; and 
p30 compressing said unmasked/ port ions to provide a second 

gi file corresponding to graphical portions of the document and 

032 storing said color information, and said first and second 

g3 files as a composite file. / 

m ^ _ 

L. 1 3,1\ A reading mactaneV comprising: 

3 2 a computer, said dompjuter comprising: 

si 3 a processor;/ y N — 

%^ a computer moaltdr for displaying the image 

5 representation of a docjiment having text and graphical image 

6 content ; 

7 a mass storage device, said storage device including 

8 software comprising instructions for causing the computer 

9 to: 

10 decompress a file into an image file and text 

H file; J 

12 Reconstruct the document from the 

13 decompressed image file and text file and apply 

14 color to the text. in accordance with color 



- 61 - 



15 information and positional information prdvided^ 

16 from .said text file; and / 

17 display the reconstructed image 

18 representation of the document pn a computer 

19 monitor and manipulate the displayed reconstructed 

20 image representation of the /document by using 

21 positional information in t/he text file. 

?=> / 

1 3/2. A reading machine system/ of claim 31 further 

2 comprising instructions for causing the computer to scale 

3 the positional information in the^ text file for manipulation 

4 of the image in accordance with a resolution of the image 

5 file and a resolution of the text file. 

/ ■ 

1 3I2 . A reading machine system as recited in conjunction 

2 with claim 32 wherein said program further comprises 

3 instructions for causing the' ^Lmpruter to: 

4 permit the user to select a^skocument item from a 

5 displayed reconstructed image/ Jepres^tation of the . 

6 document; and j 

7 apply the converted text representation corresponding 

8 to the document item to a speech synthesizer to cause the 

9 computer to output a synthesized speech representation of 
10 the document item. 

/ 

1 3/4 . A reading machine system as recited in conjunction 

2 with claim 33 further comprising instructions for causing 

3 the computer to display image representation of the document 

4 to be highlighted by applying a highlighting indicia to the 

5 displayed reconstructed image representation in accordance 
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with scaled positional information provided from tiie text 
file 

3/ 



ft 4 



The reading machine system of clafim 34 further 
comprising instructions for causing a computer to display an 
image representation of a document iten/to be highlighted 
with a color by applying a color to tlie displayed image 
representation in accordance with positional information 
provided from the converted text file. 

$fy / 

3%. A reading system comprising: 
a computer, said computer ^comprising: 
a processor; / 

a computer monitor for displaying an image 
representation of a document having text and graphic or 
image content information; 

a mass storage crevice, said storage device 
including a computer program product for decompressing 
a file containing image p>nformat ion .and text 
information, said prpgrany residifig on a computer 
readable medium coil^^si^g instructions for causing a 
computer to: j 

decompress a file into an image file and a 

text file; 

allocate a target bit map to represent the 
decompressed file; 

insert the decompressed image information 
into the /target bit map at locations specified by 
information contained in said file; and 

insert text information into said target bit 
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map in accordance with positional information 
provided from the decompressed text vfile. 



41 



The reading system of claim 3 y further comprising 
instructions for causing a computer to/display the 
reconstructed representation of the decompressed file on a 
computer monitor. 

*)P j& . The reading system of claim 36 further comprising 
instructions for causing a computer to fill the target 
output bit map with a color corresponding to a dominant 
background color provided from dolor information in the 
file. 



The reading system as recited in conjunction with 



claim 38 further comprising instructions for causing a 



computer to apply a color 
target bit map in accord 
provided from said file. 



the text information in the 
color information 





*N/7 4®. A compter program product for detecting a crease 
in an input color^image of /a document, said software product 



disposed on a compu 
instructions for 

retrieved data 



er readable medium comprising 



caus: 



/ 



cor 



g /a computer to: 

iing to portions of the page 



that divides the page intersections; 
and for each one of the sections 

retrieve a one^dimension^l .array of pixels with a 
length equal tp/the width of the\portion; and 



- 64 - 



S' 



10 determine for each of the one-dimensiojialarray of 

11 pixels whether the array has an intensify minima, by 

12 measuring a distance from the cen^r of an assumed intensity 

13 minima out to a dominant background color. 




1 £§JsWy f 1 ' The compiler pro-am product of claim 4 0 wherein 

2 for any slices that contaip no/dominant background color 

3 pixels in the appropriate direction such samples are 

4 considered to be invaJL^/d and are discarded, 

^0 y, 

1 4^2. The computer program product of claim 41 wherein 

2 for those samples that have, a dominant background color at 

yy 3 the appropriate location, a Venter for the crease is 

W \ 

..^4 determined by averaging the centers of the best slices. 



s in ji 

U 1 • The computer . program product of claim 42 wherein 

dp \ 

* 2 the first average of the centers \>f all the valid slices are 



sorted by increasing distance from\the first average and the 
M;4 average is recomputed using only the centers of the highest 

y35 (NSLICES/2) +1, where (NSLICES) is the number of slices, 

6 4#. The computer program product\of claim 43 wherein 

7 the whole area is considered to be invalid if there are less 

8 than (NSLICES/2) +1 valid slices. 

1 tytJfcy a£ 9 The computer program product of claim 44 wherein a 

\ \ 

2 composite width is assigned for the area crease as the 

> \ 

3 mxnimum area slice width, and a composite vector of 

4 intensities for each slice is constructed from the center 

5 point of the crease to the near dominant background color 



\ 
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point . for v the slice, 



T^e .computer program product of claim 45 wherein 
an array corresponding to the composite vector of 
intensities for each slice is filled in as follows: 
for a "center" area; 

define slifee(I) to be the pixel in a slice that is I 
number of pixels^.from the center in the direction of the 
near-DBC point; and 



fill in the alrrray; 



array [i] \= average of intensities of the slice [i] 
^pixels for the valid slices; and 
iterate over I £rom the center out to the near-DBC 
point as: \ 

array [i] = maximum of array [i] and array [i - 1] 
for each side of the crease, producing two arrays. 

4>Ub>*M^ The comput:er program product of claim 45 further 

comprising the step of; 

assigning a quality to\each area of the page with the 
quality being equal to the wUdth of the crease found or an 
invalid crease indicator if the area/crease fails to qualify 
as a crease 

if there are less than (NSLICES/2) +1 valid slices, or 
the width is below a minimum crease width, or if the 
majority of centerpoints used to^construct the average 
centerpoint are not within a constant horizontal distance or 
one anothefr ^or if the vector of intensities appears concave. 



^ 3^. The computer program product of claim 47 wherein 
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the crease with the highest quality is determined as the 
crease fors^the page, 

en *\i 

\ 4^. A computer program product for removing a crease 

stored on a computer readable media, comprises instructions 
for causing a computer to: 

set all pixels to the outside of the center portion of 

a left or right sVde of the image crease to a dominant 
background color. >\ 

ft/ x ^ ^ 

/ jh^^^/ > V mputer P ro 9 ram product of claim £JBT further 

comprising instructions for causing a computer to: 

set all pixei^s within a fixed distance to the dominant 
background color unless the width of the crease is less than 
a predetermined values; and 

bleach all other pixels within the width of the crease 
between the center point\and the near dominant background 
color point, 

V \ * 

^L. The\computer program of. claim &e" wherein the 
instruction forNcausing the computer to bleach comprises 
instructions for causing the computer to: 

define array [ij^Nto be the intensity in the creases f s 
intensity vector at a distance I pixels from the center; 

define image [y] [x] ^S^ 56 tiie P^ xe l i n the image x 
pixels horizontally and y pixels vertically from the upper- 
left corner; 

define center to be the center of the crease and width 
to be its width; 

define intensity (pixel) to be\a\f unction that returns 



the intensity of a pixel; 

for aVLeft-side crease, iterate over y, for each row in 
the image, iYerate over I from a fixed distance over crease 

width: \ 

if ( (intensify (image [y] [center + I] + (intensity (DBC) - 

arrayM] ) ) ) > (0 . 90 * intensity (DBC) ) ) 
\\ 

set image [y]W:enter + I] to DBC and set the 

correspond ing pixeYs in the B/W image to white. 
^ \5~ ^ \\ 

ft ^ 

Jfi . A computer program product for detecting and 
removing a crease in an input color image of a document, 
said product disposed on a computer readable medium 
comprising instructions for causing a computer to: 

retrieved data corresponding to portions of the page 
that divides the page into sections; and for each section 

retrieve a one -dimensional array of pixels with a 
length equal to the width of the portion; 

determine for each of the one -dimensional array of 
pixels whether the array has an intensity minima, by 
measuring a distance from the center of an assumed intensity 
minima out to a dominant background color; 

set all pixels to the outside of the center portion of 
a left or right side of the image crease to a dominant 
background color; 

set all pixels within a fixed distance to the dominant 
background color unless the width of the crease is less than 
a predetermined value; and 

bleach all other pixels within the width of the crease 
between the center point and the near dominant background 
color point . 
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